3D shapes have complementary abstractions from low-level geometry to part-based hierarchies to languages, which convey different levels of information. This paper presents a unified framework to translate between pairs of shape abstractions: $\textit{Text}$ $\Longleftrightarrow$ $\textit{Point Cloud}$ $\Longleftrightarrow$ $\textit{Program}$. We propose $\textbf{Neural Shape Compiler}$ to model the abstraction transformation as a conditional generation process. It converts 3D shapes of three abstract types into unified discrete shape code, transforms each shape code into code of other abstract types through the proposed $\textit{ShapeCode Transformer}$, and decodes them to output the target shape abstraction. Point Cloud code is obtained in a class-agnostic way by the proposed $\textit{Point}$VQVAE. On Text2Shape, ShapeGlot, ABO, Genre, and Program Synthetic datasets, Neural Shape Compiler shows strengths in $\textit{Text}$ $\Longrightarrow$ $\textit{Point Cloud}$, $\textit{Point Cloud}$ $\Longrightarrow$ $\textit{Text}$, $\textit{Point Cloud}$ $\Longrightarrow$ $\textit{Program}$, and Point Cloud Completion tasks. Additionally, Neural Shape Compiler benefits from jointly training on all heterogeneous data and tasks.

Video provides us with the spatio-temporal consistency needed for visual learning. Recent approaches have utilized this signal to learn correspondence estimation from close-by frame pairs. However, by only relying on close-by frame pairs, those approaches miss out on the richer long-range consistency between distant overlapping frames. To address this, we propose a self-supervised approach for correspondence estimation that learns from multiview consistency in short RGB-D video sequences. Our approach combines pairwise correspondence estimation and registration with a novel SE(3) transformation synchronization algorithm. Our key insight is that self-supervised multiview registration allows us to obtain correspondences over longer time frames; increasing both the diversity and difficulty of sampled pairs. We evaluate our approach on indoor scenes for correspondence estimation and RGB-D pointcloud registration and find that we perform on-par with supervised approaches.

Most neural networks for computer vision are designed to infer using RGB images. However, these RGB images are commonly encoded in JPEG before saving to disk; decoding them imposes an unavoidable overhead for RGB networks. Instead, our work focuses on training Vision Transformers (ViT) directly from the encoded features of JPEG. This way, we can avoid most of the decoding overhead, accelerating data load. Existing works have studied this aspect but they focus on CNNs. Due to how these encoded features are structured, CNNs require heavy modification to their architecture to accept such data. Here, we show that this is not the case for ViTs. In addition, we tackle data augmentation directly on these encoded features, which to our knowledge, has not been explored in-depth for training in this setting. With these two improvements -- ViT and data augmentation -- we show that our ViT-Ti model achieves up to 39.2% faster training and 17.9% faster inference with no accuracy loss compared to the RGB counterpart.

Artificial Intelligence (AI) is having a tremendous impact across most areas of science. Applications of AI in healthcare have the potential to improve our ability to detect, diagnose, prognose, and intervene on human disease. For AI models to be used clinically, they need to be made safe, reproducible and robust, and the underlying software framework must be aware of the particularities (e.g. geometry, physiology, physics) of medical data being processed. This work introduces MONAI, a freely available, community-supported, and consortium-led PyTorch-based framework for deep learning in healthcare. MONAI extends PyTorch to support medical data, with a particular focus on imaging, and provide purpose-specific AI model architectures, transformations and utilities that streamline the development and deployment of medical AI models. MONAI follows best practices for software-development, providing an easy-to-use, robust, well-documented, and well-tested software framework. MONAI preserves the simple, additive, and compositional approach of its underlying PyTorch libraries. MONAI is being used by and receiving contributions from research, clinical and industrial teams from around the world, who are pursuing applications spanning nearly every aspect of healthcare.

我们提出了一个简单的基线，用于直接估计两个图像之间的相对姿势（旋转和翻译，包括比例）。深度方法最近显示出很强的进步，但通常需要复杂或多阶段的体系结构。我们表明，可以将少数修改应用于视觉变压器（VIT），以使其计算接近八点算法。这种归纳偏见使一种简单的方法在多种环境中具有竞争力，通常在有限的数据制度中具有强劲的性能增长，从而实质上有所改善。

从单个图像中识别3D中的场景和对象是计算机视觉的长期目标，该目标具有机器人技术和AR/VR的应用。对于2D识别，大型数据集和可扩展解决方案已导致前所未有的进步。在3D中，现有的基准尺寸很小，并且方法专门研究几个对象类别和特定域，例如城市驾驶场景。在2D识别的成功中，我们通过引入一个称为Omni3d的大型基准来重新审视3D对象检测的任务。 OMNI3D重新排列并结合了现有的数据集，导致234K图像与超过300万个实例和97个类别相结合。由于相机内在的差异以及场景和对象类型的丰富多样性，因此3d检测到了这种规模的检测具有挑战性。我们提出了一个称为Cube R-CNN的模型，旨在以统一的方法跨相机和场景类型概括。我们表明，Cube R-CNN在较大的Omni3D和现有基准测试方面都优于先前的作品。最后，我们证明OMNI3D是一个用于3D对象识别的功能强大的数据集，表明它可以改善单数据库性能，并可以通过预训练在新的较小数据集上加速学习。

新型视图合成（NVS）是一项具有挑战性的任务，需要系统从新观点中生成场景的影像图像，在新观点中，质量和速度对应用都很重要。以前的基于图像的渲染（IBR）方法很快，但是当输入视图稀疏时质量较差。最近的神经辐射场（NERF）和可推广的变体可带来令人印象深刻的结果，但不是实时的。在我们的论文中，我们提出了一种具有稀疏输入的可推广的NVS方法，称为FWD，该方法可实时提供高质量的合成。凭借明确的深度和可区分的渲染，它以130-1000 X的加速和更好的感知质量取得了SOTA方法的竞争结果。如果有的话，我们可以在训练或推理过程中无缝整合传感器深度，以提高图像质量，同时保持实时速度。随着深度传感器的越来越多的流行率，我们希望使用深度的方法将变得越来越有用。

一个3D场景由一组对象组成，每个对象都有一个形状和一个布局，使其在太空中的位置。从2D图像中了解3D场景是一个重要的目标，并具有机器人技术和图形的应用。尽管最近在预测单个图像的3D形状和布局方面取得了进步，但大多数方法都依赖于3D地面真相来进行训练，这很昂贵。我们克服了这些局限性，并提出了一种方法，该方法学会预测对象的3D形状和布局，而无需任何地面真相形状或布局信息：相反，我们依靠具有2D监督的多视图图像，可以更轻松地按大规模收集。通过在3D仓库，Hypersim和扫描仪上进行的广泛实验，我们证明了我们的进近量表与逼真的图像的大型数据集相比，并与依赖3D地面真理的方法进行了比较。在Hypersim和Scannet上，如果没有可靠的3D地面真相，我们的方法优于在较小和较少的数据集上训练的监督方法。

尽管电子健康记录是生物医学研究的丰富数据来源，但这些系统并未在医疗环境中统一地实施，并且由于医疗保健碎片化和孤立的电子健康记录之间缺乏互操作性，可能缺少大量数据。考虑到缺少数据的案例的删除可能会在随后的分析中引起严重的偏见，因此，一些作者更喜欢采用多重插补策略来恢复缺失的信息。不幸的是，尽管几项文献作品已经通过使用现在可以自由研究的任何不同的多个归档算法记录了有希望的结果，但尚无共识，MI算法效果最好。除了选择MI策略之外，归纳算法及其应用程序设置的选择也至关重要且具有挑战性。在本文中，受鲁宾和范布伦的开创性作品的启发，我们提出了一个方法学框架，可以应用于评估和比较多种多个插补技术，旨在选择用于计算临床研究工作中最有效的推断。我们的框架已被应用于验证和扩展较大的队列，这是我们在先前的文献研究中提出的结果，我们在其中评估了关键患者的描述符和Covid-19的影响在2型糖尿病患者中的影响，其数据为2型糖尿病，其数据为2型糖尿病由国家共同队列合作飞地提供。

我们从看不见的RGB图像提出了一种场景级3D重建，包括遮挡区域的方法。我们的方法是在真正的3D扫描和图像上培训。由于多种原因，这个问题已经证明很难;真正的扫描不是防水，禁止许多方法;场景中的距离需要推理跨对象（使其更加困难）;并且，正如我们所示，表面位置的不确定性激励网络以产生缺少基本距离功能属性的输出。我们提出了一种新的距离样功能，可以在非结构化扫描上计算，并且在对表面位置的不确定性下具有良好的行为。计算此功能在光线上可进一步降低复杂性。我们训练一个深度网络来预测此功能，并显示出于TASTPORT3D，3D前面和SCANNET上的其他方法。